Measuring the risks and rewards of drug development: New research from MIT shows that the success rates of clinical trials are higher than previously thought

The study has broad implications for investors, regulators, policymakers, and clinicians

Cambridge, Mass., January 31, 2018––A new study* by researchers at the Massachusetts Institute of Technology found that nearly 14% of all drugs in clinical trials eventually win approval from the FDA-an amount that is much higher than previous studies indicate. The researchers also found that approval rates for specific illnesses range from a high of 33.4% for infectious-disease vaccines to a low of 3.4% for cancer.The study, published online today in the journal Biostatistics, has broad implications for the ways in which investors, regulators, policymakers, and clinicians calculate the risks and rewards of investing in drug development programs. The probability of success (POS) of a clinical trial is a key input to their decisions.

"One of the main responsibilities of investors and pharma executives is risk management, hence they need to know what the chances are that a compound will transition from Phase 1 to Phase 2 to Phase 3 and, ultimately, receive FDA approval," says Andrew W. Lo, the study's senior author and director of MIT's Laboratory for Financial Engineering. "Without accurate and timely estimates, resources may be misallocated and financial returns may be misjudged, which leads to higher development costs, higher-priced drugs, and lost opportunities for investors and, more importantly, patients."

The results of the research-the largest study of its kind to date-were derived from the Citeline® dataset provided by UK-based publishing company, Informa®. The dataset contained more than 400,000 entries corresponding to 185,994 unique clinical trials of over 21,000 compounds. To analyze such a large dataset, Prof. Lo and his colleagues Chi Heem Wong and Kien Wei Siah developed automated algorithms to trace each drug development path, infer phase transitions, and compute the POS statistics within a matter of hours. To put this in perspective, generating POS estimates using over 217,000 clinical trial entries available in the ClinicalTrials.gov database would take months to years if the matching and inferences had to be done manually.

"You can't manage what you don't measure," says Lo. "It would be literally impossible for any human to process all of these numbers by hand. But in the Era of Big Data, we can do this quickly and accurately."

The study sheds new light on the common perception that drug approval rates have declined over time. The research team found that while approval rates did fall between 2005 and 2013, they have been trending upward since then. For example, oncology drugs had the lowest overall approval rate of 3.4%; in 2015, however, this estimate increased to 8.3%, no doubt partly due to the recent progress in immuno-oncology. 

The study's findings also have important consequences for the future design and implementation of clinical trials. For instance, the study provides POS estimates for trials that do and do not stratify patients using biomarkers, a genetic sequence that causes, or is associated with, disease. The researchers found that trials using biomarkers tend to be more successful than those that do not. This points to the growing role of companion diagnostics in improving clinical trial success rates.

According to Prof. Lo, the study is part of an ongoing effort to help make the biomedical ecosystem more efficient. "We hope to provide this information on a regular basis-it's not just a one-shot deal," he says. "As clinical trial success rates improve for certain diseases, it's likely that more investment capital will flow into those areas. For diseases where success rates stall, public policy can play an important role by increasing research funding or providing more incentives to risk-tolerant investors and philanthropic organizations.

"It's kind of like the difference between driving with GPS today versus driving 20 years ago when maps and friends were the only navigational tools at our disposal," Lo observes. "Our goal is to show all stakeholders the lay of the land so that they can make more informed decisions about where and how to direct their resources."

* "Estimation of clinical trial success rates and related parameters," Biostatistics doi.org/ 10.1093/biostatistics/kxx069, by Chi Heem Wong, Kien Wei Siah, and Andrew W. Lo, MIT Laboratory for Financial Engineering, Sloan School of Management, Computer Science and Artificial Intelligence Laboratory, and Department of Electrical Engineering and Computer Science, Cambridge, MA